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ABSTRACT 



The invention relates to a method for the processing of a 
multidimensional image which is formed by points repre- 
senting objects of different intensity and are delimited from 
a background by way of their edges; this method includes 
steps for the segmentation (100; 30, 40) of the image into 
cells belonging to the background and cells belonging to the 
objects, for the application of criteria which impose that said 
cells (C ( -) have substantially uniform intensities, that the cells 
neither intersect nor enclose any object edge, and that the 
object edges coincide with cells edges; this method includes 
a step for the presegmentation (30) of the image into a first 
number of cells while performing a classification of the 
image points in the various cells as a function of intensity 
gradients of these points, and an actual segmentation step 
(40) for reducing the first number of cells by classifying the 
cells in the form of pairs of adjoining cells (Cy, C^, by 
evaluating a merger threshold (^ 0 (j,k)) for the cells of the 
pairs as a function of their intensity variance (cr 2 ) and of 
their intensity gradients (||G||), by evaluating a specific 
merger threshold (Iq 77 ) below which a predetermined number 
(Hq 7 ) of adjoining cells of pairs is merged, and by perform- 
ing the concomitant merger of adjoining cells of pairs whose 
merger threshold is below said specific threshold. 

15 Claims, 6 Drawing Sheets 
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IMAGE PROCESSING METHOD 
INCLUDING STEPS FOR THE 
SEGMENTATION OF A 
MULTIDIMENSIONAL IMAGE, AND 
MEDICAL IMAGING APPARATUS 
UTILIZING THIS METHOD 

FIELD OF THE INVENTION 

The invention relates to a method for the processing of a 
multidimensional image which is formed by points repre- 
senting objects which have an intensity which differs from 
one object to another, and are delimited from a background 
by way of their edges, which method includes steps for the 
segmentation of the image. The invention also relates to a 
medical imaging apparatus for carrying out the segmentation 
method, notably to an X-ray apparatus. 

The invention is used in medical imaging systems for the 
three-dimensional display of the vascular system. 

BACKGROUND OF THE INVENTION 

Herein, a multidimensional image is to be understood to 
mean, for example, an intensity image formed by points 
which are pixels arranged in a two-dimensional matrix in 
space, the intensity of the pixels being a third dimension, or 
an intensity image formed by points which are voxels 
arranged in a three-dimensional matrix in space, the inten- 
sity of the voxels being a fourth dimension. The segmenta- 
tion in an image processing method determines segments 
which are regions of different intensity of the image. The 
boundaries of the segments are called interfaces. 

An image processing method for performing segmenta- 
tion of an image is already known from the publication "A 
Multiscale Algorithm for Image Segmentation by Varia- 
tional Method" by G. Koepfler, C. Lopez, J. M. Morel in 
"SLAM J. Numer. Anal., Vol. 31, No. 1, pp. 282-299, 
February 1994". The method described in the cited publi- 
cation includes the processing of a two-dimensional image 
which initially comprises regions formed by pixels which 
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and are plotted in the original image^ According. to-. this: 
algorithm, when the term linked to the length of the inter- 
faces has a high cost, there is a tendency to eliminate the 
interface of the two adjoining regions causing such a high 
cost, provided that the intensity variance between these 
regions is small, thus inviting the merging of the two 
adjoining regions in order to minimize the Energy function. 
Because the aim is to merge the regions while starting by 
using the lowest value of the scale parameter X, at first the 
interfaces are not severely treated, so that numerous adjoin- 
ing regions having a very similar intensity are allowed to 
preserve their interface for a long time. Subsequently, as the 
value of the scale parameter X increases, the regions are 
merged two by two in a slow and gradual manner until the 
Energy function cannot be minimized further. This algo- 
rithm holds for several image scales, hence the names scale 
parameter or pyramidal parameter )u 

It is a major problem that the segmentation method known 
from the cited document is very costly in respect of calcu- 
lation time, because the regions are merged in a very gradual 
manner while utilizing first of all the lowest possible value 
of the parameter X which is subsequently increased slowly 
step by step, so that the resultant regions, must be reorga- 
nized after each merger, involving a new calculation of the 
terms of the Energy function, so as to attempt anew the 
minimization of the Energy function and, occasionally, to 
execute another merger. 

An even more significant problem is that it cannot be 
ensured that a merger between two adjacent regions which 
have substantially the same intensity but nevertheless belong 
to different objects will be avoided. 

SUMMARY OF THE INVENTION 

It is an object of the invention to provide a segmentation 
method for a multidimensional image which is less consum- 
ing in respect of calculation time than the method known 
from the cited document. 

It is another object of the invention to provide such a 



are delimited by interfaces and a given number of which has 40 segmentation method which ensures that on the one hand a 



substantially the same intensity. To this end, the method- 
includes a step in which a segmentation algorithm is applied. 
The proposed algorithm aims to eliminate the largest pos- 
sible number of interfaces in order to merge adjacent regions 
whose intensities are practically identical. This algorithm 
uses a cost function called Energy. The merging of two 
adjacent regions is possible only in the case in which the 
Energy function is minimized. This Energy function com- 
prises two terms: a first term which takes into account the 
intensity variance in each region of the image and a second 
term which takes into account the total length of the bound- 
aries in the image, weighted by a so-called scale parameter 
X. The execution of the algorithm consists first of all in 
assigning the value 1 to the scale factor X and in merging two 
adjacent regions, if any, which minimize the Energy func- 
tion. The resultant regions are then re-organized by elimi- 
nation of the interface of the two merged regions, the terms 
of the Energy function are calculated again and a new 
attempt for a merger is made, utilizing the scale factor X=l. 
This operation is repeated until there is no longer any region 
having an adjacent region for a merger when the scale factor 
X=l. After each merger the resultant regions are 
re-organized by elimination of the interfaces. Subsequently, 
the same operations are performed with the scale parameter 
X=>2, etc., un til the Energy function cannot be further mini- 
mized For"completion the interlaces ot the adjoining 
regions of different intensity are concretized by way of edges 



background can be distinguished and that on the other hand 
different objects can be distinguished from one another, 
regardless of their intensity level relative to one another and 
relative to the background. 
45 These objects are achieved and the described problems 
are solved by means of a method for the processing of a 
multidimensional image which is formed by points repre- 
senting objects which have an intensity which differs from 
one object to another and are delimited from a background 
50 by way of their edges, which method includes the steps of 
segmenting the image into cells relating to the background 
and cells relating to the objects, and of applying criteria 
which impose that said cells have respective, substantially 
uniform intensities, that said cells neither intersect nor 
55 enclose any object edge, and that the object edges coincide 
with cell edges. 

In one application, the step for the segmentation into 
object cells and background cells in this image processing 
method is preceded by steps for the acquisition of two- 
60 dimensional (2-D) intensity images of a volume, and for the 
reconstruction of a noisy three-dimensional (3-D) image, the 
step for the segmentation of the three-dimensional image 
into object cells and background cells being succeeded by 
steps for the reconstruction of a three-dimensional image by 
65 way of a method which applies renditions of volumes where 
each object is attributed the respective individual intensity of 
the corresponding cell. 
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- . In another application, a medicafcimaging apparatus com- : . -.^acquisition; (11)* of a . given .number* of urwo-dimensionaKd_c 

prises a system for the acquisition of a sequence of images X-ray images, referred to as 2-D projections; these 2-D 

and an image processing system which has access to the data projections, one of which is illustrated in FIG. 7 A, are 

of the images and to a system for the display of images and the output signals of an X-ray image acquisition system 

includes a processor for carrying out such an image pro- 5 which rotates 180° in a plane of rotation around a 

cessing method which includes such image segmentation volume formed by a part of the body of a patient, such 

steps as the brain, and constitute 2-D images of the vascular 

In one application of this method, the multidimensional system, 

image is a medical vascular image. subtraction (12) of the background applied to the 2-D 

... . .,m projections, including: 

In a specific application, an X-ray apparatus is associated a fot of these projectioos at given angles of 

with such a medical imaging apparatus. ^ ^ Qf ^ {Q ^ Qn q{ a 

BRIEF DESCRIPTION OF THE DRAWINGS medium into the vascular system of the patient, a 

second acquisition, at the same angles, without con- 

The invention will be described in detail hereinafter with ^ trast me dium, and subtraction of the intensities of the 

reference to the accompanying diagrammatic drawings; corresponding projections by means of markers in 

therein: order to extract the background, resulting in 2-D 

FIGS. 1A and IB show diagrams with functional blocks DSA (Digital Subtracted Angiography) projections, 

representing the steps of a method for the reconstruction of reconstruction (13) of the 3-D image of the vascular 

a segmented multidimensional image; 2Q system of the brain of the patient by combining the 2-D 

FIG. 2 shows a diagram with functional blocks represent- DSA projections in a suitable manner by means of 

ing the presegmentation sub-steps of the image reconstruc- backp rejection methods known to those skilled in the 

tion method; art. 

FIG. 3 shows a diagram with functional blocks represent- Referring to FIG. 7A, it appears that the 2-D projections 

ing the actual segmentation sub-steps of this method; 25 make it difficult for the radiologist to distinguish the various 

FIGS. 4A, 4B and 4C illustrate the formation of catch vessels of the vascular system. Referring to FIG. 7B, the 3-D 

basins and the elimination of watershed lines during the displayed, utilizing a known technique which forms 

presegmentation step; a rendition of voluminal surfaces, is better suitable for 

FIGS. 5A and 5B show regions before and after merger, makin S * diagnosis^More specifically while corn- 
respectively, in a two-dimensional image during the merger 30 paring the FIGS. 7A and 7B it appears that the ^shaped 
ste lT 7 vessel at the extreme left in the 3-D image of FIG. 7B is the 

, . , — , , visible reconstruction of the vessel which is practically 

FIGS. 6A and 6B show a simple histogram and an - A - 4 - 4 . * a. 1 r» • c 

. , . . . \ c , . . . indistinct in the corresponding zone of the 2-D image of 

accumulated histogram, respectively, of the weighting pjfj 7A . Moreover, such a ^luminal image enables the 

parameter of the cost function; , . . . u . \„ . . 4 . 0 , 

r 7 35 radiologist to travel m space, i.e. in the vascular system. 

FIGS. 7A, 7B, 7C show a 2-D medical projection image, ^ 2 _ D DSA projections shown in FIG. 7A are noisy; 

representing a part of the vascular system of the brain, a 3-D therefore, the reconstructed 3-D image shown in FIG. 7B is 

image reconstructed on the basis of several 2-D projections also noisy ^ u due ^bXy to the fact that the acquisition 

obtained without segmentation treatment, and a 3-D image system ^ not perfecl) that the distribution of the contrast 

reconstructed after segmentation; ^ mcdium varies during the acquisition, that the registration of 

FIG. 8 shows a medical X-ray imaging apparatus. the two projections in order to extract the background is not 

DESCRIPTION OF THE PREFERRED perfect and that the 3-D recoratruction algorithm itself is 

EMBODIMENTS perfect. The imperfections of the image of FIG. 7B are 

a nuisance to the radiologist wishing to evaluate the pres- 

The invention relates to an image processing method for 4S ence or the extent of anomalies, 

the reconstruction of a three-dimensional image, referred to Referring to FIG. 1A, therefore, the reconstructed 3-D 

as 3-D and having a fourth, intensity dimension, from a image should preferably undergo two optimization phases, 

noisy 3-D image by means of steps which include noise being: 

suppression in said 3-D image, presegmentation and seg- a first phase 14 for reducing the noise, 

mentation for separating a uniform background rendition 50 a second phase 100 for performing the segmentation, 

from objects having different intensities which, however, first phase 14 for noise reduction utilizes methods 

may be substantially close to that of the background, and for wmcn are known to those skilled in the art and will not be 

separating these objects from one another. The steps of this described herein. The second phase 100 utilizes a segmen- 

image processing method are illustrated in the form of process for differentiating and subsequently separat- 

diagrams with functional blocks in the FIGS. 1A, IB, 2A, ss ^ regions, containing exclusively the essential infor- 

2B,and 3. The invention can be generalized to images which ma tion constituted by the vessels in the envisaged 

have one or more spatial dimensions with an intensity application, and second regions which contain the whole 

dimension; such images are called multidimensional images. remainder of the image elements, that is to say everything 

In an example as illustrated in FIG. 7B, the intensity that is not a vessel and is considered to form the background, 

image to be processed is a three-dimensional vascular medi- 60 The first and second separated image regions are called 

cal image, so an image formed by voxels having three spatial "segments". The segmentation phase 100 thus serves to 

co-ordinates, a fourth dimension being formed by the inten- provide an optimized 3-D image which shows the vascular 

sity of the voxels. In the envisaged application, the image system on a uniform background. 

may contain 128 3 voxels. A specific problem occurs during this segmentation phase 

Referring to FIG. 1A, the three-dimensional image 3-D is 65 100 because the 3-D image to be segmented may contain on 

produced by a reconstruction (10) of vascular images, the one hand vessels with a high intensity and on the other 

including the following sub-steps: hand vessels with a lower intensity which deviates only 
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slightl>i:from..jthe:badcgromid.iDtensity.. Therefore, u\is-.not,j. . surface^ rendition method wbLcb)(.faithfully^reprodnoes; t all'. 

possible to segment the image into exclusively a class vessels with their different intensities, dimensions and 

"vessels" and a class "background", for example by apply- shapes on a uniform background. 

ing a simple thresholding operation to the intensity of the Referring to FIG. 2, the presegmentation step 30 includes 

voxels. Such a thresholding operation could have several 5 a determination of elementary cells as shown in FIG. 5A in 

detrimental consequences. First of all, such a thresholding a noise -suppressed 3-D image yielded by the step 14, which 

operation might eliminate vessels or details of vessels which cells do not have a predetermined shape and satisfy the 

have an intensity which deviates only slightly from that of previously defined first condition that no elementary cell 

the background but are nevertheless essential specific struc- should contain nor intersect vessels or edges of vessels and 

tures which are different from the background. Moreover, 1(J the second condition concerning the uniformity of intensity, 

such a thresholding operation might eliminate the small An essential problem thus concerns the localizing of the 

details of vessels or the vessels of very small diameter, exact positions of walls of vessels. 

because the edges of vessels are not necessarily fines or In the case of exhaustive, ideal and non-noisy data, the 

surfaces of the same intensity, so that given parts of these modulus of the intensity gradient, evaluated in a 2-D or 3-D 

edges could drop below the selected threshold and be 15 image, differs from zero (0) if this intensity gradient is 

eliminated, so that on the one hand the dimensions and on evaluated in the vicinity of a wall of a vessel. This property 

the other hand the shapes of vessels in the reconstructed 3-D may be of assistance in resolving the problem of localizing 

image might be modified. walls of vessels. However, the solution to this problem is not 

The present invention proposes a segmentation method simple because, in the case of a 3-D image, a wall of a vessel 

100 which enables an attractive reconstruction of the 3-D 2Q is defined as a set of voxels where the modulus of the 

image, in which: gradient reaches a local maximum along its own direction, 

all vessels are separated from the background and are Referring to FIG. 2, a suitable solution for solving this 

preserved with their own intensity in the same 3-D problem and for carrying out the presegmentation operation 

image, irrespective of their different intensity levels 30 involves a specific method for the classification of voxels, 

relative to the background in the image to be seg- 2S including a transformation of the 3-D image into a topo- 

mented; this is very important so as to make a medical graphic image. 

diagnosis of vascular insufficiency; In me case of a P lane or 2 " D image, it is easy to represent 

the dimensions, in particular the diameters, and the shapes sucb m ""^fiC as a topographic image having three spatial 

of vessels are reproduced very faithfully; this is very dimensions, the first two spatial dimensions x, y then being 

important for making a medical diagnosis of the pres- 30 mose of ^ 2 " D P lane ima S e while me ihM dimen - 

ence of anomalies of a diameter such as stenoses and S10n 15 forrned b y the intensity I(x,y) of pixels forming 

anomalies of a shape such as aneurysms. rcHe£s * Analogously, such a topographic image can also be 

Moreover, the segmentation method 100 according to the constructed by starting with a 3-D voluminal image in which 

invention requires a calculation time which is much shorter aa intensity function becomes a fourth spatial dimension for 

than that required by the known method. For example, the 35 me formation of reliefs. In a 3-D image, the moduli of the 

segmentation method with a reconstruction of the 3-D image gradients form surfaces of a topographic image that can be 

according to the invention can be carried out while using represented and the variations of the moduli of the gradients 

SPARC or ULTRASPARC machines which are known to form the relie£s > vaUevs CB or summits WS of the 

those skilled in the art and execute the calculations neces- topographic image. FIG. 4A shows such a 3-D topographic 

sary for the reconstruction of a segmented 3-D image of 40 ma & m a sectional view taken along the axis of x. This 

128 3 voxels in approximately one minute. topographic image contains, for example a mountain M with 

Referring to FIG. IB, the segmentation method 100 a summit WS and valleys CB, on which imaginary rainwater 

comprises two general steps: flowing along the flanks of the mountam M collects, to both 

a presegmentation operation 30 which consists in deter- ?j dc ? of ^ mit whi <* * wa *^d fine, 

mining in the 3-D image a plurality of elementary, 45 ^ h vaUe J CT to Sld f S u of Xh * ^T U ™* B a 

non-geometrical cells which are defined by a first Mrographic ^or so-called catch basin. Cateh b asms CB are 

condition which excludes that an elementary cell con- 6 f n f as independent classes and a method is defined for 

tains or intersects a vessel or an edge of a vessel, and me edification of the voxels of the image whereby two 

by a second condition which includes that an elemen- voxels ^ deemed to belong to the same class, i.e. to the 

tary cell contains exclusively voxels of substantially 50 Same C u atch baSiI ! CB ' * ^ of wa * r faUin S 

identical intensity, oato mese voxels on a surr - ace of the topographic image 

_ , ' An . . . , - reach the same minimum of said surface, 

an actual segmentation operation 40 in which pairs of A ,. 4 . . , . n . 4 , , it _ , , . , 

. . 1 . 11 e 4U . j • According to this classification method, the voxels which 

adjoining elementary cells of the pre segmented image * j . • j *uu - • * j 1 

J J. J . .1. j • j * * are situated outside any catch basm, i.e. outside any class, 

are combined by a merging method in order to form ... 4 . * . . ,. vJo ... , . 

l4 t „ 1« 0 . tU & + j *,- .u . , are situated on the watershed lines WS, so on the boundaries 

resultant cells while preserving the first condition that 55 r A . , , - , , 4 . , . . T . . , Al 

l4 4 „ .A ,°. . 4 of the classes defined by the catch basins. It is subsequently 

a resultant cell neither contains nor intersects any , . 4l _ » 4l _ . J . „ „ - . . j 

. , c 1 j • j j... tL ; decided that the elementary cells C of the presegmented 

vessel or edge of a vessel, and the second condition that . TT> „ . _/ . , 4 * , L , . 

. A & „ . , * c i_ * * ■ 11 image JPS shown in FIG. 5 A are identical to the catch basins 

an elementary cell contains only voxels of substantially ^ 

equal intensity, the merging operation being continued * - t „ . . 4 , 

V, . J . ■ u r 11 uuu Referring to FIG. 2, in practice the presegmentation step 

until the image contains a number of cells which has 60 „ rt . . . w * r 

been reduced in comparison with the presegmented 30 mcludes the sub-steps of: 

image, said number being chosen as necessary and determining^), at each voxel, the modulus of the 

sufficient for separating the vessels from one another gradient ||G||, a gradient vector ~f of this modulus of 

Referring to HO^S'steps are followed by a third 65 A ^ ^°t and its modulus |7jfc 

general step 60 in which (see FIG. 7Q an optimized 3-D determining (32) a class of catch basins CB by following, 

image is reconstructed, for example by means of a voluminal at each voxel, the direction of the gradient vectors ~f of 



M'< ■-.i-sv.vxj jy ;> ' 1 t.^-v 
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■^.i^ma* w^u. ^'^'the g&ckehts 0^||untitt lo^ 

modulus of the gradient || (j || is found, the set of voxels 
converging towards the same local minimum Min||G|| 



of the modulus of the gradient ||G || being regrouped as 
belonging to the same catch basin CB; 
determination (33) of a class of watershed fines WS: the 

voxels whose gradient moduli of the gradient \\~f\\ 
cancel one another without corresponding to local 10 

minima Min||Cj|| and which cannot be assigned to any 
catch basin CB are thus assigned to the watershed lines 
WS; 

numbering (34) the catch basin CB from 1 to n, denoted is 
as CB a -CB„; 

distributing (35) the voxels of the watershed lines WS in 
the nearest catch basin CB in such a manner that there 
are no longer boundary lines formed by voxels but 
instead catch basin interfaces are found which are 20 
called BD and extend between pairs of voxels VBD 
which are formed by a voxel in each of the adjoining 
catch basins; as appears from the FIGS. 4B, 4C, the 
voxels of the watershed lines WS are simply attributed, 
during the alternating scanning SCN, to the catch 25 
basins situated to both sides of the interface BD, 
deciding (36) that the resultant catch basins, comprising 
the voxels of the catch basins CB plus the distributed 
voxels of the watershed lines WS, are the elementary 
cells searched, having the numbers 1 to n assigned to 30 
the catch basins. 
As a result of the use of this method of classifying voxels 
and identifying elementary cells at the catch basins, neither 
the walls of vessels nor the vessels can traverse the classes 
of cells or catch basins CB, because the walls of vessels are 35 
situated at the boundaries of the classes, i.e. they coincide 
with the summits WS, so that the vessels themselves form 
independent classes. Moreover, these walls of vessels can be 
detected with the precision of detection of the watershed 
lines. This method of classification and identification thus 40 
suitably satisfies the two conditions imposed in the pre seg- 
mentation step. Referring to FIG. 5A, the image is then 
formed by elementary cells which are numbered C lf . . . 
C f , . . . C n and do not cross or contain any vessel. 

After the presegmentation step 30, the number from 1 to 45 
n elementary cells is too large for attractive reconstruction of 
a 3-D image by voluminal surface rendition. Referring to the 
FIGS. IB, 3, an actual segmentation step 40 is then carried 
out. The segmentation 40 is achieved by application of a 
merger algorithm to the presegmented image in order to 50 
minimize the number of interfaces by linking adjoining cells 
having similarity properties. This similarity is measured _by 
a cost function F wh ich comprises a sum of two terms. The 
cost function can in general be written as: 



oterms.of the cost function constitute criteriarfor reducing the.' 
number of cells by merger, subject to the conditions that the 
intensity of the voxels within the merged cells is substan- 
tially uniform and that no cell resulting from the merger 
contains or traverses vessels. 

The first term is a criterion which takes into account the 
intensity differences of cells resulting from the presegmen- 
tation and is formed by a summing function of a function 
f^m^iC^] of the variances cr^Q) of the intensities in the 
various cells C p where the variances are multiplied by the 
number m, of voxels in the cell Q considered. The variance 
d\C^ is calculated in each of the cells while taking the 
mean value of the square of the standard intensity deviation 
with respect to the mean intensity in the cell. For ease of 
evaluation of the first criterion, the intensity within the cells 
is rendered substantially uniform by individually calculating 
the mean intensity \^ OY in each cell of the presegmented 
image and by assigning this mean intensity to each of the 
voxels of the cell considered. The cost function can thus be 
written equivalently as follows: 

F=Yj y ) " WU +(ll<7ll/ll<7 0 [|) 2 ] (2b) 



55 



F=£/i[«rto)l+A£/a(ilGil) 

C; VfiD 



(1) 



and specifically as: 

F= ^m^cc-j+A^ +(I1C||/ISCo!]) 2 ] W 

C; VBD 

( iwo^dioining cells can be merged exclusively if this 
merger results in minimization of the cost function F. The 



60 
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in which the variance function ^(cr 2 ) in the first term is the 
function for summing the squares of the differences between 
the intensity I(x,y) of a voxel having the co-ordinates x, y 
and the mean intensity \? fOY of the cell C,- whereto this voxel 
belongs. It will be recalled that upon a merger the intensity 
variance of the cell resulting from the merger is always 
greater than the sum of the intensity variances of the cells 
before the merger (statistical rule). Therefore, this criterion 
disfavors any merger but it is counterbalanced by the second 
term. 

The second term is a criterion which is weighted by a 
weighting factor X and is formed by a function for summing, 
over all pairs of voxels situated at the interface of the cells, 
of a specific measure f 2 (||G|[), taking into account the modu- 
lus of the gradient ||G|| at each of the voxels of these pairs at 
the interface, ||G 0 || being a normalization factor. According 
to the second criterion, in the adjoining cells the sum 

£/2<HCil), 



taking into account the moduli of gradients ||G|| already 
calculated, is formed for the pairs of voxels VBD situated to 
both sides of an interface BD. 

For a fixed given value of the weighting factor X, this 
second criterion constitutes a penalty which is more severe 
as the moduli of the gradients of the voxels of the interface 
between two cells are smaller. This second criterion thus 
stimulates the merger of the cells whose interfaces exhibit 
small gradient moduli because, by way of this merger, the 
penalty term is minimized, so that the cost function F tends 
towards minimization, even while taking into account an 
increase of the intensity variance due to the merger. On the 
other hand, this second criterion applies a small or zero 
penalty when the voxels of the interface of two adjoining 
cells exhibit strong gradient moduli so that there is no 
stimilus to merge these cells, taking into account the fact that 
the intensity variance which increases upon the merger 
prevents minimization of the cost function F. Thus, the 
separation of the classes of vessels and background can be 
preserved, because it is known that the modulus of the 
gradient has a high value at the edges of the vessels. 
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* - -This*type: of ccst-function;for jpecformingiinergers. is not- 
known from the publication cited as the state of the art. 
According to the invention, the first, presegmentation step 
determines, on the basis of the watershed lines WS, inter- 
faces BD in very exact locations. The second, segmentation 5 
step selects, on the basis of the specific cost function F, on 
the one hand the parts of the interfaces BD which are truly 
edges of vessels having an exact location, and on the other 
hand the various parts of vessels, each of which constitutes 
a separate class which is suitably separated from classes 
forming the background by the interfaces BD which coin- 
cide with the edges of vessels. Therefore, no class whatso- 
ever can intersect a vessel and no edge of a vessel can be 
eliminated during a merger. 

^ During the actual segmentation step 40, mergers of 
adjoining cells are performed two by two when the voxels of 
these cells have characteristics which are taken into account 
by the two criteria, being such that the cost function F is 
minimized by the merger. If the cost function F is not 
minimized but increases due to the merger, such a merger is 
not performed. 

It is to be noted that the second criterion of the cost 
function F is also weighted by a variable weighting factor X. 
In order to carry out the segmentation operation 40 by means 
of the cost function F, th_e_ p arameter X must be gradually 25 
increased as from an initial value X^. For example, if ttie 
value Mistaken to equal zero, only the first term will be left 
in the function F and it will be evident that there is no 
interest whatsoever in merging two cells, because the vari- 
ance of a merged cell is always larger than the sum of the 30 
variances of two individual cells (statistical rule). 

The cells having been numbered in advance for X^O, all 
cells C 2 to C„ which adjoin one another and hence form pairs 
are visited. For each pair the function F operates a compe- 
tition between the first term, relating to the variance, and the 35 
second term, relating to the inverse of the modulus of the 
gradient of the voxels of the interface, weighted by the factor 
X. As has already been stated, the first term, concerning the 
variance, increases in the case of merger whereas the second 



20 



which .-include ,< in orden .to reduce >.the* calculation uime ; ; the ^ 
formation of a first, simple histogram on the basis of the 
evaluation of thresholds for merging pairs of two adjoining 
cells in order to provide the number of pairs of cells which 
are merged with the same given threshold, and the formation 
of a second, so-called accumulated histogram, on the basis 
of the first, simple histogram, in order to supply the number 
of pairs of cells which merge with a merger threshold lower 
than or equal to a specific threshold. In order to illustrate this 
description, merely by way of example it is assumed that at 
the end of the presegmentation 30 20,000 classes with 
50,000 interfaces have been obtained in the presegmented 
3-D image. 

Theoretically speaking, the segmentation 40 should 
include the following sub-steps: 

evaluation of the cost function F before any merger, then 
being called F 0 : in the first term of F 0 the summing 
function is calculated for the 20,000 cells resulting 
from the presegmentation, and in the second term the 
summing function is calculated for the 50,000 
interfaces, 

selection for examination of a pair of two cells C,-, 
evaluation of the cost function F after the merger of the 
two cells of said pair, the cost function then being 
called Fi: in the first term of F 3 the sum is formed for 
the 20,000 cells minus one, and in the second term the 
sum is formed for the 50,000 interfaces minus one, 
determination of a merger threshold Xo(j,k) on the basis of 
the weighting parameter X which constitutes the thresh- 
old at which the interface between the two cells of the 
pair examined disappears and enables the merger of the 
two cells of the pair examined, said merger producing 
the cost function F a :\)(j,k) is defined as the value 
which enables cancellation of a function Eg calculated 
as the difference between the cost functions F± and F 0 , 
in which the weighting parameter is XqQ^c); this is 
written as: 



) 



Etf-F^FoRO with k~kJjX) 



(3) 



term decreases because of the fact that the interface of 40 Consequently, Xo(j,k) can be readily calculated for two 
merged cells is suppressed, said cells not forming more than 
one, and hence the number of voxels for which the second 
term must be calculated decreases. For a pair of given 
adjoining cells the competition between the first and the 
second term of the cost function F may yield a minimization 45 
of the cost function F and hence a merger of the cells of this 
pair, taking into account not only the intensity variance 
function and the gradient modulus function, but also the 
variable value of the weighting parameter X of the second 
term. For a given number of cells produced by the 50 
presegmentation, the first term of the cost function F is fixed, 
so that as the value of X is smaller, the possibility of 
minimizing the cost function F and of merging the cells is 
smaller, whereas the value of X is larger, the greater the 
possibility of minimizing the function F and merging the 55 
cells will be. 

After each merger, the intensity of the resultant cell is 
calculated as the mean value of the intensities of the voxels 
of the two merged cells and the pairs of voxels of the 
interface of the two merged cells become normal internal 60 
voxels. 

It is to be recalled that it is one of the objects of the 
invention to execute the segmentation operations within a 
period of time which is shorter than that currently known 
from the state of the art. Therefore, the actual segmentation 65 
step 40, utilizing the cost function F, is carried out while 
using specific sub-steps which are described hereinafter and 



specific cells Cy, C^:# * 
Act/-*) = mj m k (I? or - ir r ) Z j(mj + 1 /[l + (M)/ IIColl] < 4 > 



where \j MOY and l k MOY are the mean intensity values and m ; -, 
m* are the numbers of voxels in the cells. 

In practice the actual segmentation step 40 does not 
include the evaluation of a cost function F in conformity 
with the formules (1) or (2), because this calculation is too 
long. The segmentation step 40 is based on the evaluation of 
the merger threshold X 0 (j^c) and includes the following 
sub-steps: 

determination (41) of pairs of adjoining cells C ft C k in the 

presegmented image; there are as many interfaces BDy A 

as there are pairs; 
determination (42) of mean intensities l/* oy , \ k MOY and 

numbers m^, m* of voxels of adjoining cells Cy, C k of 

the pairs; 

determination (43) of as many merger thresholds X^Q^c) 
as there are pairs of adjoining cells C,, utilizing the 
formule (4); 

classification (44) of the thresholds with XqQ ,k) from the 
smallest to the largest, denoted as X^ to \^ tAX \ 

formation of a first diagram (51) h^ (see FIG. 6A) which 
is called a single histogram and is based on the fact that 



^ F. - F» 

F . < ^ 
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. i niiL practice, a^givea, number- of*;pairs. of . cells; show ,the»* • i. , • . » .-^between -the, remaining cells-arefno.longerj:lhe;same^asrt 
same first term in the evaluation of the cost function F those between the initial cells; the simple histogram 51 
and the same second term, with the result that the cells and the accumulated histogram 52 are updated and the 
of these pairs may be merged for the same threshold: on slice 1^/ containing the desired number of occurrences 
the abscissa the values of the thresholds Xq 1 to Xq** 4 *, 5 is then chosen in order to merge a new percentage of, 
being numbers which can in theory lie between 1 and for example 10 or 15% of the pairs of cells; this 
50,000 if there are 50,000 interfaces, are plotted and on operation continues until an appropriate number of 
the ordinate there is plotted the number of the event h p , cells has been reached; it can be elected to save, for 
that is to say the number of times h^, that interfaces are example 10% or 15% of the initial cells: in the remain- 
found which produce Eq=0, where Xo P is from Xq 1 to jo mg ceUs th e voxels of eliminated interfaces are normal 
X/^and where l^piMAX. The number of couples internal voxels and the intensities are the mean inten- 
that can be merged with X/ is called h*; h is the sines of merged adjoining cells, 
number of couples that can be merged with A</\ and Referring to the FIGS. IB and 7C, the segmented 3-D 
h MAX is the number of pairs that can be merged with image can then be subjected to a reconstruction operation 60 
\ i MAX ; the number hj^^, corresponding to 'k 0 MAX J is 15 with voluminal surface rendition, so that it can be displayed 
not necessarily large; it is not excluded that hj^^; it or recorded. As a result of the segmentation according to the 
appears that for given values of X/*, no feasible merg- invention, each of the vessels is reproduced with a specific 
ers are found (1^=0) whereas for other values of X^ a intensity level which enables the vessels to be distinguished 
large number h p of feasible mergers of pairs is found. from one another and from the background. 
Therefore, upon the formation of the simple histogram 2 o FIG * * sn o ws a digital radiography apparatus which 
several thousands of values of Xq p may occur on the includes an X-ray source 1, a table 2 for accommodating the 
abscissa; in order to make the calculations simple and patient, and an image intensifier device 3 which is coupled 
fast, the values of X^ are rearranged in slices of, for to a camera tube 4 which applies data to a digital image 
example 256 values and these slices are actually plotted processing system 5 which includes a processor for carrying 
on the abscissa in said simple histogram 51; the values 2 s out me image processing method according to the invention. 
Xq 1 to Xq 256 are thus rearranged in a slice called Iq 1 , the The processor includes several outputs, one output 6 of 
values X^ 256 to X^ 11 being rearranged in a slice Iq 2 , and which is coupled to a monitor 7 for the display of radio- 
so on until a final slice Iq""" is formed; in each slice the graphic images, or to an appropriate recording system 8. The 
numbers of events h^, are added and referenced analo- image processing system may form an integral part of the 
gously as ho 1 to ho"" 1 *; thus, actually the values are 30 X-ray apparatus or be installed in a remote location while 
plotted on the ordinate in this simple histogram 51, being coupled thereto. 

formation (52) of a second, so-called accumulated histo- Generally speaking, the described image processing 

gram H/ (see FIG. 6B) which is based on the simple method can be used, partly or completely, by any medical 

histogram and in which the slices determined in the imaging system, notably by X-ray apparatus such as 

simple histogram 51, from Iq 1 to Iq""" are plotted on the 35 described above for the processing of multidimensional 

abscissa and in which the accumulated values in each images, with or without reconstruction of voluminal sur- 

slice, from ho 1 to ho"""", are plotted on the ordinate, that faces. 

is to say the numbers of events added from the first slice All references cited herein are incorporated herein by 

until a slice considered; thus, in the slice Iq 1 a number reference in their entirety and for all purposes to the same 

of events ho 1 , now called Hq 1 is arranged; in the slice 40 exteat ^ tf eaca individual publication or patent or patent 

V a number of events h^+ho 2 , now called HoVis application was specifically and individually indicated to be 

arranged, and so on until the slice l^ in which a incorporated by reference in its entirety for all purposes, 

number of events is arranged which is equal to the sum What is claimed is: 

of ho 1 to ho™^ or U 0 max f 1- A method for the processing of a multidimensional 

selective segmentation (53) of the 3-D image: in order to 45 whicn * formed b Y P° mts representing objects which 

perform a segmentation of the 3-D image at a high have 311 intensity which differs from one object to another, 

speed, on the basis of the accumulated histogram (52) and m delimited from a background by way of their edges, 

it is chosen, for example to perform the segmentation mc method comprising: 

in steps, in each step a given number of concomitant segmenting the image into cells relating to the back- 
mergers being executed in order to eliminate a given 50 ground and cells relating to the objects, by applying 
percentage of interfaces from a section, for example criteria which impose that said cells have respective, 
10% or 15% of the interfaces; to this end, in the substantially uniform intensities, that said cells neither 

i accumulated histogram 52 t here is select ed a slice 1/ intersect nor enclose any object edge, and that the 

which contains the desired number of eveffisTtfiisvaTu^ object edges coincide with cell edges, 

of 1/ cancels the above function Eq and hence mini- 55 2 - A method as claimed in claim 1 wherein the segmenting 

^mizes the cost function F for all pairs of cells having further comprises 

produced a merger threshold X^^k) in this slice; all presegmenting the image into a first number of cells while 

pairs of adjoining cells are visited and the cells are performing a classification of the points of the image in 

mer ged whenever J hey correspond to a mer^rT^resrT- the various cells in conformity with a criterion which is 

oldlLfiJ^lower tha^^ 60 a function of intensity gradients (\\G\\) of said points, 

thus^nabling the~eTJecution~oi ffie~mergers of these and 

pairs concomitantly with a first elimination of the actually segmenting so as to reduce the first number of 

desired number of interfaces; subsequently, the remain- cells by 

ing cells are reorganized by calculating the intensity of classifying the cells into pairs of adjoining cells (C ; -, C^) 

the resultant cells as the mean value of the intensities of 65 while evaluating a merger threshold (X^jk)) of the 

the original cells; for the resultant cells the terms of the cells of the pairs as a function of the intensity 

relation (4) are calculated again, because the relations variance (o 2 ) and the intensity gradients (]|G|[) in the 
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; .i w. .c;-* cells ■? of.e ach »pak. • andr while^evaluating- -a ^specific- 
merger threshold (l^ below which the adjoining 
cells of pairs are merged in a predetermined number 
(Ho(p)) and by 
performing the concomitant merger of adjoining cells 
of pairs having intensity characteristics which lead to 
a merger threshold which is lower than the specific 
threshold. 

3. A method as claimed in claim 2 wherein the preseg- 
menting further comprises transforming the multidimen- 
sional image into a topographic image by way of intensity 
gradient functions of the image points while identifying (Q) 
the cells searched in catch basins (CB) and while identifying 
the edges bounding said cells at watershed lines (WS) of the 
topographic image. 

4. A method as claimed in claim 3 wherein the points of 
the watershed lines (WS) are collected in the adjoining catch 
basins (CB) which are situated to both sides of these lines, 
wherein the cells obtain a uniform intensity which is calcu- 
lated as the mean value (1/*°*) of the intensity of the points 
within the catch basins and form a class of objects, and 
wherein interfaces (BD) are defined between pairs of edge 
points (VBD) of the pairs of adjoining cells and coincide 
with object edges. 

5. A method as claimed in claim 2 wherein the actually 
segmenting further comprises: 

defining a cost function (F) which comprises a first term 
which is a function of the intensity variance (o 2 ) within 
the cells present in the image, and a second term which 
is weighted by a variable weighting factor (X) and is a 
function of intensity gradients at the edge points of 
cells, and 

evaluating the merger threshold (X^(j,k)) in order to 
perform the merging of two adjoining cells (C,, C^) of 
a pair as the value of the variable weighting factor (X) 
which cancels the difference (Eq) between the cost 
function (F 0 ) calculated before the merging of two cells 
and the cost function (Fj) calculated after this merger. 

6. A method as claimed in claim 5 wherein the cost 
function is given by the formula: 

f= £ [/(X, Y)-^) 2 +a£ t/[i + (HOII/HColl) 2 ], 
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in which the first term is the intensity variance function, 
expressed by a first function for the summing, over all cells 
(Q), of the squares of the differences between the intensity 
(I(x,y)) of a voxel having the co-ordinates (x,y) and the 
mean intensity (l t MOY ) in the cells (Q) whereto this voxel 
belongs, and in which the second term is weighted by the 
weighting factor (X) and is formed by a second function for 5S 
the summing, over all pairs of voxels situated at the interface 
of the cells, of a specific measure which takes into account 
the modulus of the normalized gradient Q|G||/]|G 0 |D of the 
voxels of these interface pairs, and 

wherein the merger threshold O^QJc)) enabling the 
merger of two adjoining cells (C y , Qt) to be performed 
is evaluated as a function of the differences of the mean 
intensities Qf*° Y , \ k MOY ) of two cells to be merged, and 



60 



7. A method as claimed in claim 5 wherein evaluating the 
specific merger threshold (Iq 75 ), enabling the concomitant 
execution of several mergers of two adjoining cells (C ; -, C^), 
further comprises: 

determination of merger thresholds (Xo(j,k)) for the cells 
of all pairs of cells of the image, 

classification of these values (Xo(j,k)) of the merger 
threshold according to increasing values (X^/% the 
number of said values being at the most equal to the 
total number of interfaces present in the presegmented 
image (lSV^Vr"* 

determination of slices (l^) of these values of merger 
threshold factors classified according to increas- 
ing values, 

formation of a simple histogram of the number (h 0 '5 of 
merger events for adjoining cells in each respective 
slice of the merger threshold values (X 0 (j,k)), 

formation of an accumulated histogram while inscribing 
in each slice (If/ 7 ) the accumulated sum (IV) of the 
number of said events (h Q ip ) from the first slice (L^) 
until the slice considered (l^, for all slices (Iq 1 1 q j ^^ 
of the merger threshold values (X^O, and 

definition of said specific merger threshold as the slice 
(lo P ) which comprises the number of events (IV) 
corresponding to a predetermined number of concomi- 
tant mergers. 

8. A method as claimed in claim 7 further comprising, 
after evaluating the specific merger threshold, 

selecting a predetermined number of interfaces to be 
concomitantly eliminated between adjoining cells of 
pairs, 

visiting the pairs of adjoining cells of the image and 
merging adjoining cells of pairs whenever the merger 

threshold relating to these cells (\>G»k)) is below said 

specific merger threshold QJ*). 

9. The method of claim 3 wherein the actually segmenting 
further comprises: 

defining a cost function (F) which comprises a first term 
which is a function of the intensity variance (o 2 ) within 
the cells present in the image, and a second term which 
is weighted by a variable weighting factor (X) and is a 
function of intensity gradients at the edge points of 
cells, and 

evaluating the merger threshold (X^jk)) in order to 
perform the merging of two adjoining cells (C p Q) of 
a pair as the value of the variable weighting factor (X) 
which cancels the difference (Eq) between the cost 
function (Fq) calculated before the merging of two cells 
and the cost function (FJ calculated after this merger. 

10. The method of claim 9 wherein the cost function is 
given by the formula: 

F = £[/(*, Y)-? 0Y } 7 i/U +(M/IICoH) 2 ], 



in which the first term is the intensity variance function, 
expressed by a first function for the summing, over all cells 
of the number of points (m y , in these cells, and of 65 (Q), of the squares of the differences between the intensity 
a second surriming function in conformity with a for- (I( x >y)) °* a voxe * having the co-ordinates (x,y) and the 
mula: mean intensity (l/" or ) in the cells (Q) whereto this voxel 
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wsa^vbelongsy. and- in which the^ second, te nn is . weighted-by . the ,v m 
weighting factor (X) and is formed by a second function for 
the summing, over all pairs of voxels situated at the interface 
of the cells, of a specific measure which takes into account 
the modulus of the normalized gradient (||G|1/||G 0 |D of the 5 
voxels of these interface pairs, and 

wherein the merger threshold (XqO ,k)) enabling the 
merger of two adjoining cells (C, Q) to be performed 
is evaluated as a function of the differences of the mean 
intensities 0/ VfOr , l k MOY ) of two cells to be merged, and 1Q 
of the number of points (m,, m^) in these cells, and of 
a second summing function in conformity with a for- 
mula: 

AoO-Jt) = m / im(#f OT -iF^f/imj +mi)£ +aiC0)/l|CoBl- IS 



11. The method of claim 10 wherein evaluating the 
specific merger threshold (i 0 p ), enabling the concomitant 
execution of several mergers of two adjoining cells (C /t Q), 20 
further comprises: 
determination of merger thresholds (\>(j,k)) for the cells 

of all pairs of cells of the image, 
classification of these values (Xo(j,k)) of the merger 
threshold according to increasing values (X^, the 25 
number of said values being at the most equal to the 
total number of interfaces present in the presegmented 
image (lSX/S^o""), 
determination of slices (l^ of these values of merger 
threshold factors Q^f), classified according to increas- 30 
ing values, 

formation of a simple histogram of the number (ho* 7 ) of 
merger events for adjoining cells in each respective 
slice (lo* 7 ) of the merger threshold values (^(j^k)), 

formation of an accumulated histogram while inscribing 35 
in each slice (l^ the accumulated sum (Hq 7 ) of the 



a- numbe r. of* said -events, (ta^jfrom :the first-slice*. (1^) .v^; * 
until the slice considered (1qO» for aU slice s (Iq 1 
of the merger threshold values Q^), and 

definition of said specific merger threshold as the slice 
(V) w bich comprises the number of events (H^) 
corresponding to a predetermined number of concomi- 
tant mergers. 

12. A method as claimed in claim 1 further comprising, 

prior to segmenting, acquiring two-dimensional (2-D) 
intensity images of a volume and reconstructing a noisy 
three-dimensional (3-D) image, and 

subsequent to segmenting of the three-dimensional image 
into object cells and background cells, reconstructing a 
three-dimensional image by means of a method utiliz- 
ing voluminal renditions where each object is attributed 
the respective individual intensity of the corresponding 
cell. 

13. A method as claimed in claim 1 wherein the multidi- 
mensional image is a medical vascular image. 

14. A medical imaging apparatus comprising: 

an image acquisition system for providing data of an 
image, and 

an image processing system which has access to the data 
of the image and to an image display system and 
comprises a processor for carrying out an image pro- 
cessing method as claimed in claim 1. 

15. A medical imaging apparatus as claimed in claim 14 
wherein the image acquisition system further comprises an 
x-ray source. 
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